Skip to content

moq-mux: decouple importers from the catalog, split byte-parsing into per-codec splitters, and make importers pure frame publishers#1749

Merged
kixelated merged 31 commits into
devfrom
claude/moq-mux-import-export-api-dtkqdp
Jun 18, 2026
Merged

moq-mux: decouple importers from the catalog, split byte-parsing into per-codec splitters, and make importers pure frame publishers#1749
kixelated merged 31 commits into
devfrom
claude/moq-mux-import-export-api-dtkqdp

Conversation

@kixelated

@kixelated kixelated commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

The import half of the moq-mux refactor. It lets moq-mux fill a single track on demand without a BroadcastProducer/CatalogProducer, separates each codec's byte parsing from its publisher, and removes the manual sync() footgun.

This started as "decouple the importers from the broadcast catalog" and grew (same branch, by request) to land the per-format splitters, Published-owns-decode, and pure-publisher importers that were originally deferred. Ports every single-track importer (opus, H.264, H.265, AV1, VP8, VP9, AAC, and the legacy MP2/AC-3/E-AC-3 importer) plus the dispatchers and the TS container that embeds them. All existing callers keep working.

1. The catalog bridge (publish module)

  • Renditions trait: an importer exposes the hang::Catalog it publishes.
  • Published: mirrors an importer's renditions into a catalog::Producer and retires them on drop. Generic over the catalog extension E so it can attach to a container's extended catalog.
  • unique_track(broadcast, suffix): mints a legacy single-codec track at hang::container::TIMESCALE.

Each importer is new(TrackRequest) (on-demand) / from_track(TrackProducer) (broadcast push / fixed track) with a local hang::Catalog and a lazy catalog() (eager for audio). No catalog::Producer, no Drop hook. There is no "fixed track" concept: a changed codec config re-mirrors the rendition in place rather than erroring.

2. Per-codec splitters + pure-publisher importers

Byte parsing and publishing are fully separated:

  • codec::h264::Split / h265::Split / av1::Split: dumb byte->frame engines. They find access-unit / temporal-unit boundaries, flag keyframes, and stamp wall-clock timestamps for stdin. They own no track, catalog, or config. h264/h265 cache SPS/PPS(/VPS) and re-insert them ahead of each keyframe; h264's Split handles both avc1 (length-prefixed) and avc3 (Annex-B) shapes; AV1 carries the sequence header inline. decode_stream (unknown boundaries) / decode_frame (one AU) / decode_from / seed / reset.
  • codec::{h264,h265,av1}::Import are pure frame publishers: they take already-split frames via decode(impl IntoIterator<Item = Frame>) (the FrameDecode trait) and resolve the catalog config from the inline SPS/sequence-header in the first keyframe (or an out-of-band avcC/av1C via initialize). A keyframe that can't be configured is an error.
  • The Framed/Stream dispatchers, the TS container, moq-cli, moq-rtc, and moq-video own a Split and drive split.decode_X(buf) -> import.decode(frames).
  • Splitters are independently unit-tested (h265/av1 had no tests before; h264 gained avc1 split tests). VP8/VP9 stay one-buffer-per-frame (no streaming, no split).

3. Published owns decode (the sync() footgun is gone)

  • FrameDecode + Published::decode(impl IntoIterator<Item = Frame>): the frames path; it syncs the catalog after decoding.
  • Published::decoding(|inner| ...): the byte-path/edit wrapper (still used by VP8/VP9 and the TS jitter edit), which also syncs.
  • Published::sync is private — the only way to decode is a path that syncs.

4. lenient_start -> MissingKeyframe

The container Producer no longer silently drops pre-keyframe frames. Writing a non-keyframe with no open group returns MissingKeyframe; importers write deltas straight through, and the TS/FLV containers swallow MissingKeyframe so a mid-stream join resumes cleanly at the next keyframe.

5. thiserror everywhere

The single-track importers no longer surface anyhow: vp8/vp9/legacy (and the ac3/eac3/mp2 header parsers) get thiserror Error enums wired into crate::Error via #[from], matching h264/h265/av1.

TS container

H.264/H.265/AAC/legacy per-PID streams build through Published + a per-PID Split. AAC's synthesized description is set on the importer before attach (so Published::new's mirror covers it) and the audio-burst jitter refinement goes through decoding. External behavior is unchanged — the byte-exact roundtrip tests guard it.

Notes

  • Targets dev: breaking changes to moq-mux public APIs (new Split types, FrameDecode, Published::{decode,decoding}, private sync, importers lose their byte methods, removed with_lenient_start/FixedTrackReconfigured), built on dev-only primitives (TrackRequest, TrackInfo::with_timescale, Frame.duration).
  • Also fixes a pre-existing missing .await in rs/moq-ffi/src/test.rs (superseded by dev's tokio::join! fix after the merge).

Test plan

  • cargo test -p moq-mux (269 pass): on-demand + broadcast paths, lazy video catalogs, eager audio, Published sync/retire-on-drop and auto-sync via decode/decoding, the splitter packaging + boundary + keyframe-detection tests (h264 avc1/avc3, h265, av1), in-place reconfiguration, MissingKeyframe on a delta-before-keyframe, TS byte-exact roundtrips (incl. mid-stream / dirty joins), fMP4/MKV roundtrips.
  • cargo clippy --all-targets -- -D warnings, cargo fmt --all --check, RUSTDOCFLAGS=-D warnings cargo doc clean.
  • moq-cli, moq-rtc, moq-video, hang, moq-boy, libmoq, moq-ffi build.
  • moq-gst not built here (missing gstreamer-1.0 system lib); consumes the unchanged Framed API.

(Written by Claude)

claude added 2 commits June 16, 2026 02:40
First slice of the import refactor. The opus importer no longer holds a
`catalog::Producer` and mutates a shared catalog with a `Drop` hook. It now
produces frames on a single track and exposes its own standalone `hang::Catalog`
rendition, which a new `publish` bridge merges into a broadcast catalog
(removing it on drop).

- `codec::opus::Import`: `new(TrackRequest, Config)` (on-demand) and
  `from_track(TrackProducer, Config)` (broadcast push / fixed track). `decode`
  now takes `impl IntoIterator<Item = Frame>`; `decode_buf` is the raw-packet
  convenience that stamps a wall clock when no timestamp is given. `catalog()`
  returns the standalone `hang::Catalog`.
- `publish` module: `Renditions` trait, `Published<I>` (merges renditions into a
  `catalog::Producer`, retires on drop, derefs to the importer), and
  `unique_track` (mints a legacy single-codec track at the microsecond timescale).
- Rewire `import::Framed`, moq-gst, and moq-rtc through the bridge.

Tests cover both construction paths (TrackRequest and broadcast unique_track),
frame delivery, the catalog merge, and retire-on-drop.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Second slice of the import refactor. The H.264 importer joins opus on the
request-based core: it no longer holds a `catalog::Producer` or mutates a shared
catalog from a `Drop` hook. It produces frames on a single track and exposes its
own `hang::Catalog`, which the `publish` bridge mirrors into a broadcast catalog.

Unlike opus, H.264's catalog is lazy (avcC for avc1, the first SPS for avc3) and
refines over time (jitter), so `Published` gains a `sync()` that re-mirrors the
importer's renditions and is a no-op when nothing changed. Callers invoke it
after each decode.

- `codec::h264::Import`: drop the `E`/`catalog::Producer`/`TrackProvider`
  coupling. `new(TrackRequest)` (on-demand) and `from_track(TrackProducer)`
  (broadcast push / fixed track), `with_mode`, lazy `catalog() -> Option`,
  `Renditions` impl. avc1 reconfiguration now errors instead of minting a new
  track (a single fixed track can't represent a new init segment); avc3 SPS
  changes still update the rendition in place.
- `publish::Published`: generic over the catalog extension `E` (so it attaches
  to a container's extended catalog), plus `sync()` for lazy/updating catalogs.
- The TS container builds its per-PID H.264 stream through `Published`
  (`unique_track` + `from_track`), syncing after each decode; external behavior
  is unchanged (byte-exact roundtrip tests guard it).
- Rewire `import::{Framed,Stream}`, moq-cli, moq-video, moq-rtc through the
  bridge. Drop the now-unused `TrackProvider::set_suffix`.

Also fixes a pre-existing missing `.await` in the moq-ffi tests
(`dynamic_track_request_can_publish_media`) that broke `cargo check
--all-targets` on dev, unrelated to this change.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
@kixelated kixelated changed the title moq-mux: decouple the opus importer from the broadcast catalog moq-mux: decouple the single-track importers (opus, H.264) from the broadcast catalog Jun 16, 2026
claude added 2 commits June 16, 2026 03:32
…Option

moq-video's `Producer::track()` now returns `&TrackProducer` (the avc3 track is
always created eagerly), so the `.expect(...)` no longer applies. This broke
`cargo check --all-targets` on moq-boy.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Completes the import reshape: h265, av1, vp8, vp9, aac, and the legacy
audio importer (MP2/AC-3/E-AC-3) now follow the same request-based core as
opus/h264. None hold a `catalog::Producer` or mutate a shared catalog from a
`Drop` hook; each produces frames on a single track and reports its own
`hang::Catalog`, attached to a broadcast catalog through `publish::Published`.

- Each importer: `new(TrackRequest)` / `from_track(TrackProducer)`, a local
  catalog, lazy `catalog() -> Option` for the video codecs (config known on the
  first key frame / SPS) and eager `catalog() -> &hang::Catalog` for aac.
  Reconfiguration on the fixed track is an error (no new-track minting).
- `import::{Framed,Stream}`: every codec arm mints via `unique_track` +
  `from_track` + `Published`, syncing the video codecs after each decode.
- TS container: H.265, AAC, and legacy per-PID streams build through
  `Published`. AAC's synthesized `description` and audio-burst `jitter` are set
  via a `pub(crate) aac::Import::rendition_mut` + `sync`, since the rendition now
  lives in the importer's local catalog. External behavior is unchanged (the
  byte-exact roundtrip tests guard it).
- Delete the now-unused `TrackProvider` (every codec mints via `unique_track`).

All codec/TS importers now share one shape; `TrackRequest` is the on-demand
entry point and `from_track` the broadcast-push one.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
@kixelated kixelated changed the title moq-mux: decouple the single-track importers (opus, H.264) from the broadcast catalog moq-mux: decouple the single-track importers from the broadcast catalog Jun 16, 2026
claude added 7 commits June 16, 2026 13:35
First slice of the splitter / unified-decode work.

- New `codec::h264::Split`: a standalone parser that turns H.264 bytes into
  `container::Frame`s plus a resolved `VideoConfig` (the NAL/AU assembly, avcC
  parsing, SPS/PPS cache, and Annex-B wall-clock all live here, independently
  testable). avc1 still errors on reconfiguration; avc3 still updates in place.
- `codec::h264::Import` now drives `Split` internally for its byte APIs
  (`decode_frame` / `decode_stream` unchanged, so the Framed/Stream/TS
  dispatchers and external callers are untouched and don't regress) and pulls
  the resolved config into its local catalog.
- New `publish::FrameDecode` trait (`decode(impl IntoIterator<Item = Frame>)`),
  the uniform decode entry point a caller drives with frames from its own
  `Split`. `Published::decode` wraps it and syncs, so the catalog re-mirror
  can't be forgotten — the footgun-free path. h264::Import implements it.

The dispatchers still use the byte API + manual `sync()`; migrating them to
`Split` + `Published::decode` (and resolving the avc1-vs-avc3 config flow on
that path) is the next slice, then rolling the same split to the other codecs.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Per review: the splitter shouldn't know what a codec config is. It just
takes a single byte stream (no out-of-band init), finds access-unit
boundaries, and packages SPS/PPS into each keyframe so every keyframe is
self-contained.

- `codec::h264::Split` is now an Annex-B stream assembler only: no avc1, no
  `VideoConfig`, no `take_config`. It exposes `decode_stream` (unknown
  boundaries), `decode_frame` (one access unit), `decode_from`, `seed`
  (prime the SPS/PPS cache from an out-of-band parameter-set buffer), and
  `reset`. Wall-clock timestamps for stdin live here.
- `codec::h264::Import` owns all config, from exactly two sources: an avcC
  handed to `initialize` (avc1, required), or the SPS the splitter packages
  into the first keyframe, scanned out of the frame here (avc3, no init
  needed). It also owns the avc1 length-prefixed framed path; the stream
  splitter is Annex-B/avc3 only, matching "if you can init out of band you
  already know frame boundaries, so you don't need the stream splitter".
- A keyframe that can't be configured (no inline SPS, no avcC/seed) is a
  hard error. A non-keyframe before the first config is tolerated: it's a
  mid-stream-join leftover that the producer's lenient start drops ahead of
  the first keyframe (preserves `survives_midstream_join` and the dirty TS
  joins).

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
The lazy-catalog codecs all did `decode_frame(...)?; sync();` by hand, an
easy-to-forget two-step. Make `Published` own the pairing so the catalog
re-mirror can't be skipped.

- New `Published::decoding(|inner| ...)`: runs a decode/edit on the inner
  importer and re-mirrors the catalog in one call. Generic over the
  closure's error so it wraps both the `crate::Result` and `anyhow::Result`
  importers. Pairs with the existing `Published::decode(frames)` (frames in
  hand) as the byte-path equivalent.
- Convert every `decode + sync` site to `decoding`: the Framed and Stream
  dispatchers, the TS H.264/H.265 streams, moq-rtc, moq-video, and moq-cli.
- TS AAC: set the rendition `description` on the importer before
  `Published::new` (its attach-time mirror covers it) and route the jitter
  refinement through `decoding`.
- `Published::sync` is now private: the only way to decode is through a path
  that syncs, so the footgun is gone.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
`Producer::with_lenient_start` silently dropped non-keyframes that arrived
before the first keyframe. Replace that implicit drop with an explicit
`MissingKeyframe` error the producer returns, and let the one caller that
wants to tolerate a mid-stream join (MPEG-TS) skip it.

- New `container::MissingKeyframe` error: `Producer::write` returns it when a
  non-keyframe arrives with no open group (was a silent drop under
  lenient_start, or a generic ProtocolViolation without it). Wired into
  `crate::Error` and `fmp4::Error` via the `Container::Error` bound.
- Drop `with_lenient_start` entirely.
- Importers now surface MissingKeyframe for a pre-keyframe delta: h264 and
  h265 (and av1) write the delta through to the producer instead of
  pre-empting with a config error, erroring early only on an *unconfigurable
  keyframe* (NotInitialized / MissingSps / MissingSequenceHeader). vp8/vp9
  already wrote straight through.
- The TS importer wraps its H.264/H.265 decode in `skip_missing_keyframe`,
  so a capture that joins mid-GOP drops the leading deltas and resumes at the
  first keyframe (preserves survives_midstream_join + the dirty TS joins).

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Mirror the h264 split: separate the Annex-B parsing from the publisher.

- New `codec::h265::Split`: a dumb Annex-B stream assembler that finds
  access-unit boundaries, caches VPS/SPS/PPS and re-inserts them ahead of
  each keyframe, and stamps wall-clock timestamps for stdin. No track,
  catalog, or codec config. `decode_stream` / `decode_frame` / `decode_from`
  / `seed` / `reset`, like h264.
- `codec::h265::Import` now drives the splitter and owns the config: it
  scans the SPS the splitter packages into the first keyframe (or a seed
  buffer via `initialize`) to fill the catalog, errors on an unconfigurable
  keyframe, and writes pre-keyframe deltas through to the producer (which
  reports MissingKeyframe for a mid-stream join). It also implements
  `FrameDecode` so a caller with its own splitter can publish frames.
- Adds the first H.265 unit tests (the splitter packaging path).

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Round out the streaming codecs: separate AV1 OBU parsing from the publisher,
matching h264/h265.

- New `codec::av1::Split`: a dumb OBU stream assembler that finds
  temporal-unit boundaries, flags keyframes (a sequence header or a
  KEY_FRAME), and stamps wall-clock timestamps for stdin. No track, catalog,
  or codec config. AV1 carries the sequence header inline ahead of keyframes,
  so unlike H.264/H.265 there's nothing to cache or re-insert; `seed` just
  prefixes leading metadata OBUs onto the next frame. The `ObuIterator` moves
  here. `decode_stream` keeps the per-OBU wall-clock timestamps.
- `codec::av1::Import` now drives the splitter and owns the config: it scans
  the sequence header the splitter packages into the first keyframe (or an
  av1C / seed buffer via `initialize`) to fill the catalog, falls back to a
  minimal config on a parse failure, errors on an unconfigurable keyframe,
  and writes pre-keyframe deltas through to the producer (MissingKeyframe for
  a mid-stream join). It also implements `FrameDecode`.
- Adds the first AV1 splitter unit tests (boundary + keyframe detection).

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
The lenient-start drop was replaced by the producer's MissingKeyframe; update
the two h264 comments that still described the old behavior.
@kixelated kixelated changed the title moq-mux: decouple the single-track importers from the broadcast catalog moq-mux: decouple importers from the catalog, add per-codec splitters, and let Published own decode Jun 16, 2026
claude added 4 commits June 16, 2026 19:47
Resolved conflicts:
- rs/moq-ffi/src/test.rs: kept dev's tokio::join! fix for the
  dynamic-track-request test (it supersedes this branch's earlier
  sequential subscribe_media fix).
- rs/moq-mux/src/container/flv/import.rs: dev's new FLV importer used
  the removed with_lenient_start(); ported it to the MissingKeyframe
  model (drop the call, swallow MissingKeyframe at the video write so a
  mid-GOP join still works).
Review follow-up. There are no fixed tracks anymore, so a changed codec
config just re-mirrors the catalog rendition instead of erroring.

- Remove the `FixedTrackReconfigured` error variant from the h264, h265, and
  av1 error enums.
- h264: drop `set_config`; avc1 (avcC) and avc3 (inline SPS) both resolve
  through one in-place `apply_config` that no-ops on an unchanged config.
- h265 / av1: `configure_from_sps` / `apply_config` update the rendition in
  place on a change.
- av1 `Split::decode_stream` / `decode_frame` take `impl Into<Option<Timestamp>>`,
  matching the h264 and h265 splitters.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Review follow-up: a library crate shouldn't surface anyhow. Port the
single-track codec importers that still used `anyhow::Result` to the crate's
thiserror-based `crate::Result`, mirroring h264/h265/av1.

- New `vp8::Error`, `vp9::Error`, and a `legacy::Error` (covering the
  MP2/AC-3/E-AC-3 header parsers), each wired into `crate::Error` via
  `#[from]` (`Vp8`/`Vp9`/`Legacy`).
- `FrameHeader::parse` (vp8/vp9), the vp9 `BitReader`, the `ac3`/`eac3`/`mp2`
  `parse_header`s, and the vp8/vp9/legacy importer methods all return the
  typed errors now; no `anyhow::ensure!`/`bail!` remain in these modules.
- Dropped the vp8/vp9 "fixed track cannot be reconfigured" bail too, so they
  update the rendition in place like the other codecs. The dispatcher test
  that asserted the old error now asserts in-place reconfiguration.
- With every importer on `crate::Result`, `Published::decoding` drops its
  generic error parameter and just takes a `crate::Result` closure.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
The importers owned a Split internally and exposed decode_frame/decode_stream;
that duplicated the splitter and kept byte parsing in the publish layer. Move
all byte parsing to dispatcher-owned splits so the importers only take frames.

- `h264`/`h265`/`av1` `Import` lose their `split` field and the
  `decode_frame`/`decode_stream`/`decode_from` methods. They're pure
  publishers now: `decode(impl IntoIterator<Item = Frame>)` (FrameDecode) +
  config resolution (from the inline SPS/sequence-header in keyframes, or an
  avcC/av1C via `initialize`) + catalog + finish/seek/track. `initialize`
  resolves config without consuming the buffer; `seek` no longer resets a
  split.
- `h264::Split` regains avc1: it's the sole h264 byte->frame engine for both
  wire shapes (framing + NALU length size only, config stays in the importer).
  `Mode`/`with_mode`/auto-detect moved here from the importer.
- The Framed/Stream dispatchers, the TS container, moq-cli, moq-rtc, and
  moq-video now own a `Split` and drive `split.decode_X(buf) ->
  import.decode(frames)`. Small `build_h264`/`build_h265`/`build_av1` helpers
  in the dispatcher encode the "import reads config, split consumes" contract.

269 tests pass (incl. the byte-exact TS roundtrips and fMP4/MKV); the split
gained avc1 unit tests.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
@kixelated kixelated changed the title moq-mux: decouple importers from the catalog, add per-codec splitters, and let Published own decode moq-mux: decouple importers from the catalog, split byte-parsing into per-codec splitters, and make importers pure frame publishers Jun 17, 2026
claude and others added 11 commits June 17, 2026 02:49
A Split is for a raw byte stream (stdin / unknown boundaries); the
known-boundary decode_frame didn't belong on it.

- Rename `decode_stream` -> `decode` on all three splits, add `flush` (emit
  the in-flight access/temporal unit), and drop `decode_frame`. `decode_from`
  flushes at EOF. The Annex-B splits (h264/h265) keep a `tail` buffer so
  `decode` fully consumes the caller's buffer (Framed's contract) while
  retaining the trailing NAL across chunks; `flush` pulls it.
- Drop avc1 from `h264::Split` entirely. avc1 (length-prefixed + out-of-band
  avcC) is not a stream and can't arrive over stdin, so `Mode`/`with_mode`/
  `initialize`/`detect_mode` and the avc1 framing leave the splitter. avc1
  becomes a free `h264::avc1_frame(data, length_size, pts)` helper.
- Framed splits its H264 arm into `Avc1 { length_size, import }` (no split,
  wraps each AU via `avc1_frame`) and `Avc3 { split, import }`. Known-boundary
  callers (Framed avc3/hev1/av01, TS, moq-rtc, moq-video) do `decode + flush`
  per unit; stdin callers (Stream, moq-cli) flush the tail at finish/EOF.

265 tests pass (incl. TS byte-exact + fMP4/MKV roundtrips, moq-rtc bitstream).

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
No caller used the splitters' decode_from async helper (the container
importers have their own). Remove it from h264/h265/av1 Split; a caller that
wants to drive a reader loops decode() + flush() itself.
A Split is just a byte stream; a dedicated "here are out-of-band parameter
sets" hook (seed) contradicts that. Remove it from all three splits.

The dispatchers' init buffer (the codec header passed to Framed::new /
Stream::initialize) is now fed to the splitter via `decode` as the leading
bytes of the stream: the importer still reads it for the catalog config, and
the splitter caches any inline SPS/PPS the same way it would mid-stream, so a
"parameter sets once up front, then bare keyframes" encoder still produces
self-contained keyframes. av1C (the out-of-band 0x81 config record) is not an
OBU stream, so it stays config-only and isn't fed to the splitter.

The two seed unit tests now exercise the same property through `decode`
(leading params + a later bare IDR -> self-contained keyframe).

265 tests pass; dependents (moq-cli/rtc/video/ffi/libmoq) build.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Merge the new `publish` module into `import` and rename `Published` to
`import::Track`. The module is now a directory: `import/mod.rs` (the
`Framed`/`Stream` format dispatchers) plus a private `import/track.rs`
(the catalog-bridge `Track`, `Renditions`, `FrameDecode`, `unique_track`),
re-exported flat. The `publish::Published` stutter is gone, and "import"
already names the ingest direction (the mirror of `export`).

Add `moq_net::TrackDemand`, a cloneable, weak-backed watch-only handle
(`name`/`used`/`unused`/`closed`) obtained from `TrackProducer::demand()`.
It can't publish or close the track and doesn't pin the group cache, so
callers can gate on subscriber demand without holding a writable producer.
`TrackProducer: Clone` is left intact for now.

Encapsulate the high-level front door: `Framed` no longer hands out a
`&TrackProducer`. The match is now a private `producer()` helper behind
curated `name()` / `subscribe()` / `demand()` accessors. moq-ffi's
`MediaProducer` holds a `TrackDemand` instead of a cloned producer. The
low-level `Track` and codec importers keep their public `track()`, since
their callers build the track themselves, so moq-video/boy/audio/rtc/cli
need no changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reshape the import layer so each codec importer owns its catalog rendition
and deals only in whole frames, and so concurrently-produced tracks share
one timeline.

Catalog: add `catalog::VideoTrack` / `catalog::AudioTrack`, scoped handles
(via `Producer::video_track` / `audio_track`) that publish one importer's
rendition and retire it on drop. This replaces the `import::Track` wrapper
plus the `Renditions` / `FrameDecode` traits, which are deleted; the
`Published`-style mirror is gone.

Importers: every codec importer is now `Import<E: CatalogExt = ()>` built
with a single `new(track, catalog, [config])` (the `TrackRequest`
constructor and `from_track` are dropped; the on-demand path accepts the
request at the call site). They expose `demand() -> TrackDemand` instead of
handing out a `TrackProducer`, take whole frames as `&[u8]` (no more `Buf`),
and the per-importer `decode_buf` / `pts` helpers collapse into one
`decode`. `Framed::decode_frame` becomes `Framed::decode(&[u8], pts)`.

Sync: the wall-clock fallback moves to a `Clock` owned by the shared
`catalog::Producer`, so audio and video synthesizing timestamps anchor to
the same epoch (`Producer::timestamp` / `VideoTrack`/`AudioTrack::timestamp`).

Containers and consumers (ffi/cli/rtc/gst/video/boy) are updated to the new
constructors and `demand()`.

Follow-ups: split `Framed` into `Framed` + `FramedTrack`, and give the
TS/MKV/FLV/fMP4 containers their own `Split` modules so they too deal in
frames.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`import::Framed`/`import::Stream` each mixed single-codec tracks with
multi-track containers, so `demand()`/`name()`/`new_with_track` had to
fail at runtime for containers (the `MultipleTracks` error). Split them
along the multiplicity axis instead, keeping the framed-vs-stream axis:

- `Track` / `TrackStream`: one codec on one MoQ track. `demand()` /
  `name()` are infallible; `Track::from_track` replaces `new_with_track`.
- `Container` / `ContainerStream`: a container that may publish several
  tracks. No single-track handles. Both wrap one private `ContainerImpl`.

A format only appears in a `*Stream` enum if it can recover frame
boundaries from a raw byte stream, so streamability is now expressed in
the type. Formats are typed per axis: `TrackFormat`, `TrackStreamFormat`,
and `ContainerFormat` (reused for the container stream until a
non-streamable container, e.g. RTP, arrives).

`Error::MultipleTracks` is gone. Callers that take an arbitrary format
string (libmoq, moq-ffi stream) dispatch into a small track/container
enum.
The byte-parsing entry points took `T: Buf + AsRef<[u8]>` purely so they
could advance the caller's buffer past consumed bytes — but the codec
splitters and the ts/mkv/flv containers already copy their input into an
internal scratch and retain the partial tail there, so the bound bought
nothing except a generic that can't be a trait object.

Make every decode take `&[u8]`:

- Splits (h264/h265/av1) and the four containers drop the generic. av1's
  splitter gains the internal `tail` the others already had (its
  ObuIterator used to leave the partial OBU in the caller's buffer); fmp4
  gains an internal buffer + `drain`, mirroring mkv/flv, instead of
  advancing the caller's `Buf`.
- The import layer (Track/TrackStream/Container/ContainerStream) takes
  `&[u8]`; ContainerStream::initialize is dropped (containers are
  self-describing — decode already covers it).
- Callers stop threading their own buffers: moq-ffi's stream producer
  drops its BytesMut, moq-cli reuses+clears a read chunk, moq-rtc skips a
  per-frame copy, moq-srt/moq-video/moq-hls pass slices. moq-hls loses
  the `InitNotConsumed` check (the remainder is no longer observable; the
  is-initialized check still guards a bad init segment).

The internal NAL/OBU iterators stay generic over the internal buffer.
fmp4/ts/mkv/flv each had an async `decode_from(reader)` that looped
`read_buf` into a chunk and fed it to `decode`. Nothing called them, and
now that `decode` takes `&[u8]` the loop is trivial for a caller to write
inline. Remove them and the `tokio::io` imports they pulled in.
It only ever gated moq-hls, which used it to turn an init segment with no
moov into an early error and to re-check before each media fragment. Both
are redundant: an importer that never resolved its config errors on its
own at the first frame it can't place (the codec imports return
NotInitialized, fmp4 returns NoMoov), so the readiness probe added a
method to every importer for nothing. Remove it from the codec and
container imports and the stream dispatchers; moq-hls relies on the
decode error instead and drops its two now-unused error variants.

The vp8/vp9/fmp4 tests already assert on the catalog snapshot, so they
keep their coverage without the probe.
TrackFormat / TrackStreamFormat / ContainerFormat were a thin string
mapping that every caller immediately resolved into a `new()` call, and
moq-ffi's uniffi surface is string-based anyway, so the typed format was
pure internal indirection. Have each importer's `new`/`from_track` take
`format: &str` and parse it inline, erroring on a format it doesn't
handle.

Streamability is now encoded entirely by the type: `TrackStream::new`
accepts only the self-delimiting codecs (avc3/hev1/av01), and
`Container::new` / `ContainerStream::new` keep separate match lists so a
future non-streamable container (e.g. RTP) can be added to `Container`
alone — the role the proposed `ContainerStreamFormat` would have played.

The two classify-then-build callers (libmoq `media_ordered`, moq-ffi
`publish_media_stream`) build the track importer first and fall through
to a container on `UnknownFormat`; libmoq keeps its own `UnknownFormat`
error when neither matches. moq-gst passes string literals.
`Track::new` (mint a unique track) and `Track::from_track` (use a given
track) shared the entire codec dispatch; the only difference was who
created the track. Keep just the given-track form and call it `new`, so
the importer never owns track creation — the caller mints with
`unique_track` when it wants an on-demand track. The catalog rendition is
still registered lazily when the codec config resolves, so dropping the
broadcast parameter loses nothing.

Callers mint as needed: moq-gst and moq-ffi `publish_media` mint then
construct; `publish_media_on_track` just passes its requested track.
libmoq `media_ordered` now tries the container first so a codec format
doesn't mint a stray track before being recognized.

`TrackStream` keeps minting in `new` — it has no given-track caller, so
there's nothing to collapse.
Add `BroadcastProducer::reserve_track(name) -> TrackRequest`: a
producer-authored deferred track, the same shape as a consumer-driven
`requested_track()` but initiated by the producer. The track is
discoverable immediately; its `TrackInfo` (timescale) is set when the
importer accepts it.

`Track`/`TrackStream::new` now take a `TrackRequest` and accept it,
which is the single place a codec-specific timescale would be chosen
(today it's the legacy microsecond timescale for all of them). Callers
that minted a track now reserve one: moq-gst, moq-ffi `publish_media` /
`publish_media_stream`, and libmoq `media_ordered` (still container-first
so a codec format doesn't reserve a stray track).

moq-ffi's dynamic flow stops accepting the request eagerly: it hands back
a `MoqTrackRequest` (name / accept / abort), and `publish_media_on_track`
lets the importer accept it — which also fixes a latent bug where a
requested media track was accepted untimed and then rejected timed frames
with `TimestampMismatch`.
@kixelated kixelated enabled auto-merge (squash) June 18, 2026 21:05
dev independently reworked the moq-ffi dynamic flow (MoqTrackInfo,
Option<TrackInfo>/Option<Subscription> params, and a pending/active
MoqTrackProducer that accepts a requested track on first use). Keep that
design and adapt its importer calls to this branch's moq-mux API:

- publish_media / publish_media_stream reserve a track and hand the
  request to Track::new / TrackStream::new (container-first for stream).
- publish_media_on_track takes the still-pending request off the
  MoqTrackProducer (new take_pending) and lets the importer accept it.
- MediaProducer tracks demand (not the producer); MediaStreamProducer
  drops its buffer since the codec/container importers buffer internally.

Drops the branch's own MoqTrackRequest in favor of dev's pending
MoqTrackProducer, which covers the same "accept on first use" need.
Rather than dev's pending/active MoqTrackProducer, mirror moq-net's split:
requested_track() returns a MoqTrackRequest (wrapping moq_net::TrackRequest)
that you accept() into a MoqTrackProducer for raw writes, hand to
publish_media_on_track for media (the importer accepts it), or abort() to
reject. MoqTrackProducer goes back to wrapping a plain TrackProducer.

Keeps dev's MoqTrackInfo and the Option<TrackInfo>/Option<Subscription>
params; accept() takes Option<MoqTrackInfo> to match TrackRequest::accept.
requested_track() now returns MoqTrackRequest (not MoqTrackProducer), and
publish_media_on_track takes the request. Mirror that in the hand-written
wrappers: add a Python TrackRequest (accept/abort) and a Kotlin TrackRequest
alias, point requested_track/requestedTracks at it, and accept the request
for raw writes in the dynamic-track test.
@kixelated kixelated merged commit 6419a70 into dev Jun 18, 2026
3 checks passed
@kixelated kixelated deleted the claude/moq-mux-import-export-api-dtkqdp branch June 18, 2026 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants